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(54) Change log aggregation and optimization 

(57) A change log aggregation and optimization 
mechanism and methodology for updating and synchro- 
nizing application data and application files in a client 
device of a data transfer and synchronization system. 
The contents of a plurality of change logs reflecting the 
then current changes to the application data of the client 
device are downloaded and merged into an aggregate 
log. Instead of applying each change log as it is down- 
loaded, the contents of the aggregate log, representing 
ali changes to application data and/or application files 
recorded in prior change logs, are then applied to the 
client device to update application data and/or applica- 
tion files in the client device. 




Fie. L 



Q_ 
UJ 



Printed by Jouve, 75001 PARIS (FR) 



1 



EP1 180 890 A2 



2 



Description 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001 ] The invention relates to the transference of da- 
ta between two systems independent of the form in 
which the data is kept on the respective systems, and 
in particular to providing an efficient means of commu- 
nicating data between systems and devices. 

Description of the Related Art 

[0002] The growth of computing-related devices has 
not been limited to personal computers or workstations. 
The number of personal computing devices has grown 
substantially in both type and format. Small, hand-held 
computers carry a multitude of contact, personal, docu- 
ment, and other information and are sophisticated 
enough to allow a user to fax, send e-mails, and com- 
municate in other ways wirelessly. Even advanced cel- 
lular phones carry enough memory and processing 
power to store contact information, surf the web, and 
provide text messaging. Along with the growth in the so- 
phistication of these devices, the need to transfer infor- 
mation between them has grown significantly as well. 
[0003] With a multitude of different device types on 
the market, keeping information between the different 
devices synchronized has become increasingly prob- 
lematic. For example, if an individual keeps a calendar 
of information on a personal computer in his or her office 
using a particular personal information manager appli- 
cation, the individual would generally like to have the 
same information available in a cellular phone, hand- 
held organizer, and perhaps a home personal computer. 
The individual may additionally have a notebook com- 
puter which requires synchronizing file data such as 
presentations or working documents between the note- 
book and the office computer. 

[0004] Until now, synchronization between both doc- 
uments and personal information managers has oc- 
curred through direct connection between the devices, 
and generally directly between applications such as a 
personal information manager in one device and a per- 
sonal information manager in another device or using 
an intermediary svnc-mappinq Drogram. 
[0005] One example of this is the prevalent use of the 
3Com Palm® OS-based organizer, such as the 3Com . 
Palm® series of computing devices, which uses its own 
calendaring system, yet lets users synchronize the data 
therein with a variety of different personal information 
manager software packages, such as Symantec's ACTI 
™, Microsoft's Outlook®, and other systems. In this ex- 
ample, an intermediary synchronization program such 
as Puma Technology, tnc.'s Intellisync® is required. In- 
ieiiisync<s>is an application program whicn runs on both 
the hand-held device and the computer which stores the 



information data and maps data systems between non- 
uniform data records. 

[0006] in other cases, direct transfer between appli- 
cations such as transfer between Microsoft's Outlook® 
5 computer-based client and Microsoft's Windows CE 
■Pocket Outlook" application, is possible. Nevertheless, 
in both cases, synchronization occurs through direct 
connection between a personal computer and the per- 
sonal computing device, While this connection is gen- 
10 erally via a cable directly connecting, for example, a 
Palm® device in a cradle to the personal computer, the 
connection may be wireless as well. 
[0007] One component of these synchronization sys- 
tems is that the synchronization process must be able 
15 to delineate between when changes are made to spe- 
cific databases and must make a decision about wheth- 
er to replace the changed field. Normally, this is meas- 
ured by a change in one database, and no-change in a 
second database. In some cases, both databases will 
20 have changed between syncs. In this case, the sync op- 
eration must determine which of the two changes, which 
has been made is to "win" and replace the other during 
the sync. Generally, this determinant of whether a con- 
flict exists allows some means for letting the user re- 
25 solve the conflict. 

[0008] In a technical sense, synchronization in this 
manner is generally accomplished by the copying of full 
records between systems. At some level, a user is gen- 
erally required to map data fields from one application 
30 try another p.nri ,cp**cify w^»nh d 5 ?*? f^lrl*: i ?.ss i irr!s , d tc 
which corresponding field in a different device. Less 
mapping is required where developers more robustly 
support various platforms of applications. 
[0009] In many instances, the data tobe synchronized 
as is generally in the form of text data such as records of 
addresses, contact information, calendar information, 
notes and other types of contact information. In certain 
instances, data to be synchronized will be binary format 
of executable files or word processor-specific docu- 
40 ments. In many cases where document synchronization 
is required, the synchronization routine simply deter- 
mines whether or not the documents in question have 
changed, and uses a time-based representation to de- 
termine which of the two files is newer end replaces the 
45 older file with the newer file to achieve synchronization, 
as long as the older of the two files was in fact not 
channed. This is the model lifted in the familiar "Brief- 
case" function in Microsoft Windows-based systems. If 
both files have changed, then the synchronization rou- 
so tine presents the option of conflict resolution to the user. 
[0010] Generally, such synchronization schemes are 
relatively inefficient since they require full bandwidth of 
the document or binary file to be transferred via the syn- 
chronization link. In addition, a change log is typically 
55 generated with each synchronization operation to 
record the changes in any given data record. In a situ- 
ation wnere a large number of change logs have been 
generated (and stored in a server's memory), the se- 
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quence of downloading and applying each change log 
often results in the unnecessary download of data. A 
separate transaction is required for each occurrence of 
an item record in the change log sequence and it is pos- 
sible for the same field in a record to be updated many 
times during a single synchronization operation. 
[0011] Accordingly, there is a need for a change log 
aggregation an d optimization mechanism that will more 

the synchronization process. 10 

SUMMARY OF THE INVENTION 

[0012] The invention, roughly described, provides a 
method and apparatus for merging the contents of a plu- 15 
rality of change logs into an aggregate log in a down- 
load-and-appty sequence of a client device to update 
application data in the client device of a data transfer 
and synchronization system. Redundant changes in 
multiple change logs are eliminated and files and 20 
records of the application data can thus be synchronized 
and updated more efficiently via a single transaction. 
[001 3] In one aspect, the invention provides a method 
for updating application data in a client device of a data 
transfer which may include the steps of: downloading a 25 
first change log of a plurality of change logs from a serv- 
er system, where each of the plurality of change logs 
reflects changes to the application data; adding the first 
change log to an aggregate log; deleting the first change 
log; repeating the downloading, adding, and deleting 30 
sieps ror a next change log ot me plurality or cnange 
logs until no additional change logs exist; and applying 
the aggregate log to the application data to update the 
application data. 

[0014] The step of adding or merging the plurality of 35 
change logs to the aggregate log may further comprise 
the steps of: retrieving information for a valid item of the 
application data; updating a map of the aggregate log; 
writing the item to the aggregate log; updating the loca- 
tion of the valid item in the map; and repeating the pre- 40 
vious steps for all remaining valid items of the current 
change log being rolled into the aggregate log. 
[0015] In a further aspect, the invention comprises a 
method for combining changes in a plurality of accumu- 
lated change logs into an aggregate iog. The melnoti **3 
may include the steps of: creating an aggregate log; re- 
trieving the contents of a current change log; adding the 

CuiiieiitA of Li it* CUn tiiii Ciictt iyt; iuy iO ii ie agyieyaLe iuy, 

deleting the current change log; and repeating the pre- 
vious steps until no additional change logs exist. In par- so 
ticular, said step of adding or rolling the content of the 
change logs into the aggregate log may comprise the 
steps of: retrieving the contents of a plurality of items of 
the application data; updating a map of the items; up- 
dating the aggregate log; updating the location of re- 55 
spective items in the map; and compacting the aggre- 
ciste log if 9 compact t^rf?sho!(i is exc^ed^d. 
[001 6] In a further aspect, the invention may comprise 



an apparatus for updating the application data in a client 
device of a data transfer and synchronization system. 
The apparatus may comprise: a downloading routine for 
rteratively retrieving a plurality of change logs from a 
5 server system where each of the plurality of change logs 
reflect the then current changes to the application data; 
a merging routine for iteratively aggregating the con- 
tents of the plurality of change logs to an aggregate 

^■V* ^ F»n ^ ^.k n b •■ >. t » r J .. I .. H k + 1!, f . . ■ ■ . * » 

""""S" "**<iuiiy*. iwy UUICUVII I VSUlll IC I Ul XVtSX QUVCiy 

deleting the plurality of change logs; and an updating 
routine for applying the contents of the aggregate log to 
the application data to update the application data. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0017] The invention will be described with respect to 
the particular embodiments thereof. Other objects, fea- 
tures, and advantages of the invention will become ap- 
parent with reference to the specification and drawings 
in which: 

Figure 1 is a exemplar block diagram of a data 
transfer and synchronization system having a 
change log aggregation and optimization mecha- 
nism in accordance with the present invention. 
Figure 1 A is a flow diagram illustrating a start down- 
load and aggregate sequence of the present inven- 
tion. 

Figure 2 is a flow diagram illustrating the sequence 
of steps for adding a change log to an aggregate 
cnange log in accordance with the present inven- 
tion. 

Figure 3 is a flow diagram illustrating the steps for 
retrieving items of a change log in accordance with 
the present invention. 

Figure 4 is a flow diagram illustrating the steps for 
updating a map of a change log in accordance with 
the present invention. 

Figure 5 is a flow diagram illustrating the sequence 
of steps for adding a file item in accordance with the 
present invention. 

Figure 6 is a flow diagram illustrating the default se- 
quence of steps for adding an item in accordance 
with the present invention. 
Figure 7 is a tiow diagram illustrating the steps for 
merging fields of an item in accordance with the 
present invention. 

riguie 6 is a tiow uiagram illustrating tne steps ror 
merging fields of an collection in accordance with 
the present invention. 

Figure 9 is a flow diagram illustrating the steps for 
compacting an aggregate log in accordance with 
the present invention. 

Figure 10 is a flow diagram illustrating the steps for 
retrieving fields from a current item of a change log 
in accordance with the present invention. 
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■ DETAILED DESCRIPTION 

[0018] Fig. 1 is a generalized block diagram of an ex- 
emplary data transfer and synchronization system. In 
particular, the system includes network 1 0 that includes 
one or more storage servers 1 2, 1 4 and a management 
server 15. A plurality of client devices (e.g., home PC 
1 6, office PC 1 8, personal information Palm® computing 
device zu, and weo-enaoiea cenuiar [eiepnone £d) are 
capable of coupling to network 1 0 and exchanging syn- 
chronization information in an offline fashion via the stor- 
age servers 12 and 14, in accordance with the tech- 
niques described in U.S. Patent Application Serial No. 
09/490,550, filed January 25, 2000, U.S. Patent Appli- 
cation Serial No. 09/491 ,675 and U.S. Patent Applica- 
tion Serial No. 09/491 ,694 both filed January 26, 2000, 
and all entitled "Data Transfer and Synchronization Sys- 
tem," incorporated herein by reference. 
[0019] Client software of the present invention is in- 
stalled on both home PC 1 6 and office PC 1 8 of Fig. 1 , 
for example, and is configured to operate in conjunction 
with an operating system such as Microsoft Windows®. 
The client software, when executed, interacts with the 
various applications on the user's PCs. The user inter- 
acts with the client software and configures the software 
such that the applications are prioritized. Data is then 
extracted from the various applications, organized in a 
format independent of the particular application and de- 
vice from which the data originated, and incorporated 
into a data package. With exemplary embodiments of 
the present invention, various classes of data, including 
contacts (e.g., names, addresses, phone numbers, 
email addresses), Internet browser bookmarks (e.g., 
Netscape Navigator®, Microsoft Explorer®), calendar 
events, email messages (e.g., Inbox, Sent Items, Delet- 
ed Items), notes, tasks, and files (e.g., word processor 
specific documents, electronic presentations, spread- 
sheets, binary format of executable files) can be manip- 
ulated and synchronized between the plurality of devic- 
es. 

[0020] Consider, for example, a scenario where a us- 
er has installed the Microsoft Outlook® application on 
home PC 1 6 and the user has entered the information 
of five individuals into the Contacts folder. When in- 
structed by the user to synchronize the contents of the 
Contacts folder, the client software accesses Outlook® 
and assembles the information for five individuals in a 
DataPackCONT.DOOO. In this case, the UUID "CONT 
identifies Contacts information as the specific object and 
suffix D000 signifies that this DataPack is version a Q* 
The contacts are combined in DataPack CONT.D000 as 
a collection of five transactions, each of which is as- 
signed a unique ID number 1 , 2, ... 5. ID 1 may represent 
contact information for Jane Smith and ID 2 may repre- 
sent contact information for John Doe. In this example, 
each transaction 1 , 2, .. 5 is associated with an associ- 
aled. action, "Add." DataPack CCftTDOGO is subse- 
quently uploaded to network 10 and stored on, for ex- 



ample, storage server 1 2. 
[0021] Later, office PC 1 8 connects to network 1 0 and 
office PC 18 sends a signal to management server 15 
informing management server 15 that office PC 18 has 
5 not downloaded any DataPacks. Management server 
1 5 responds by sending a signal to office PC 1 8 indicat- 
ing that a DataPack CONTD000 of change information 
for contacts has been stored on storage server 12 since 
omce l a last connected to network i u. umce i 8, 
10 in response, sends a signal to management server 15 
requesting DataPack CONT.D000. The most recent da- 
ta package(s) stored on storage server 1 2 are identified 
(in this example, version 0 of the contact data, CONT 
D000) and DataPack CONT.D000 is then downloaded 
« to office PC 18. 

[0022] The change information in that data package, 
"Adds" of contacts in this case, is then applied to the 
pertinent application in office PC 1 8. In this example, the 
client software on the office PC is configured to synch ro- 
20 nize contacts using a Lotus Notes® application. Thus, 
the five Adds from CONT.D000 are applied to the con- 
tacts in Lotus Notes®, so that the contact information in 
home PC 16 and office PC 18 is synchronized. The of- 
fice PC 18 then sends a signal back to management 
25 server 15 indicating that office PC 18 has applied ver- 
sion °0 D of the contacts data package(s). This informa- 
tion is preferably maintained in a registry by manage- 
ment server 1 5 for each and every device that couples 
to the network to download and upload change informa- 
30 tion. 

[0023] Subsequently, the user of office PC 1 6 updates 
the contacts in Lotus Notes® and adds one or more con- 
tacts. In this example, ten new contacts are added. 
Thus, the Lotus Notes® application uploads a second 

35 data package to network 1 0, the data package including 
the ten contacts and each having the associated action, 
"Add." This data package represents more recent 
change information than the information in CONT.D000. 
The data package uploaded by office PC 1 8 is identified 

*o by a unique filename, in this example, CONT.D001. In 
addition, office PC 18 sends a signal to management 
server 1 5 confirming that CONT.D001 has been upload- 
ed to network 1 0. The registry is updated to indicate that 
office PC 18 has already applied the change information 

45 in CONT.D001 . 

[0024] Home PC 16 subsequently connects to data 
network 1 0, and the client software on home PC 16 com- 
municates with management server 15 coupled to net- 
work. 10. In particular, home PC 16 sends a signal to 

so management server 15 identifying CONT. D000 as the 
most recent version of change information the last time 
home PC 16 was coupled to data network 1 0. Manager 
ment server 1 5 queries the storage servers for any more 
recent data packages of changes to contact information. 

55 Management server 15 identifies such data package(s), 
CONT.D001 in this example, and sends a signal to home 
PC 16 informing home PC 16 iJiat such dala package 
(s) exist. The client software on home PC 1 6 then re- 
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. quests the new data package(s), and management 
server 15 then downloads the data package(s) to home 
PC 15. The change information therein, in this case the 
ten contacts from CONT.D001 to be added, is then ap- 
plied to the Contact information maintained by Microsoft 
Outlook® on home PC 16. The communication of the 
change information to Microsoft Outlook® and subse^ 
quent updates to the contacts in Outlook® are coordi- 

• • . « • **. . .»*...* . i. _ . . „ r-i^. ji - -»-• . , 

naicu uj uic uiciu ouunoic un iiuhic ii>. i nus, uic 

Contact information in home PC 16 and work PC 18 is 
again synchronized. 

[0025] Other transactions, in addition to "Add," are 
provided with exemplary embodiments of the present in- 
vention. One of these is the transaction, "Modify." Using 
the present example, the contact information for a per- 
son may sometimes change after CONT.D001 is up- 
loaded to network 1 0. For instance, Jane Smith may call 
the user on the telephone and tells the user that Jane 
has changed her phone number. The user then access- 
es office PC 1 8 and changes the phone number for Jane 
Smith in the user's Contacts folder. 
[0026] The user then activates, for example, a "Syn- 
chronize" button displayed on the computer screen by 
the client software, so a new data package or the 
change log, CONT.D002, is created and uploaded to 
network 1 0 and stored on one of storage servers 12,14. 
A signal is sent by office PC 18 to management server 

15 informing management server 15 that data package 
CONTD002 has been uploaded. Data package COISTT. 
D002 differs from data packages CONT.D000 and 
CONT.D001 , in that the action, "Modify" is used instead 
of "Add." The Modify instruction and the associated 
change information is correlated with the particular user. 
In particular, the Modify instruction is associated with the 
pertinent ID, in this case ID 1 representing Jane Smith. 
In addition, data package CONT.D002 includes the field 
to be modified, in this example, "Phone," and the new 
information, in this example, Jane Smith's new phone 
number. 

[0027] Subsequently, when home PC 16 connects to 
network 1 0, the data package CONT.D002 is download- 
ed to home PC 16, and the client software recognizes 
that, for IDj the information within the field "Phone" has 
been updated. The modification is then made to this 
contact information via Microsoft Outlook*©. Home PC 

16 then sends a confirmation signal to management 
server 1 5, confirming that home PC 1 6 has received and 
appnea tne cnange information in version wd or ine con- 
tacts data packages. The pertinent information in the 
registry for home PC 1 6 is then updated. The next time 
home PC 1 6 couples to network 1 0, home PC 1 6 sends 
a signal to management server indicating that home PC 
16 has received CONT.D002. If no subsequent data 
packages with change information for contacts have 
been stored on the storage servers, then no data pack- 
ages are downloaded to home PC 16. 
[0023] As changes arc rr.sds izr various classes of 
data, data packages accumulate on storage servers 12, 



1 4 and consume valuable storage space. As the number 
of stored data packages increases, the amount of avail- 
able storage space on storage servers 12, 14 corre- 
spondingly decreases. In the example above, assume 
5 for illustrative purposes that data package CONT.D000 
occupies 2 kilobytes ("K") of memory on storage server 
12, CONTD001 occupies 1 K, and CONT.D002 occu- 
pies 0.5 K, for a total of 3.5 K memory. In situations a 

uotri » HI I HUTU, I VI CAOJIipiC, IU 25 1 1 ItTydiJy iej> ("*fvi *) Ul 

ro memory, the amount of available storage space contin- 
ues to decrease as data packages are updated and 
change logs are accumulated, until storage space on 
the storage servers 12, 14 are exhausted. 
[0029] Thus, prior to the present invention, each syn- 
5 chronization created a unique change log to record any 
changes in the data to be synchronized. The change log 
is subsequently uploaded to network 10 via, for exam- 
ple, an Internet connection and stored in storage server 
12. Overtime, as the user performs additional synchro- 
0 nizations, the change logs accumulate and when a new 
client device, or a client device that has not been recent- 
ly synchronized, is connected to network 10 and syn- 
chronized, all or some substantial subset of the accu- 
mulated change logs are downloaded and applied one 
* at a time. Thus, if there are three change logs (e.g., a 
first change log may record a change in a contact's 
home phone number, a second change log may record 
an addition of a work phone number for the contact, and 
a third change log may yet record the deletion of the 
> contact entirely), all three change logs must be down- 
loaded and applied to the client device one at a time 
when a new client device is subsequently synchronized, 
even though the contact is ultimately deleted. 
[0030] In situations where there is a sequence of 
change logs, there may be numerous conflicting chang- 
es to the same item. When this happens the user is 
asked to decide which of the conflicting versions to 
keep. Priorto change log aggregation (i.e., with multiple 
change logs), each time an item, for which a conflict ex- 
40 isted, was changed, the user was required to resolve 
ostensibly the same conflict. In an aggregated log, there 
would generally be only one entry for each item and 
thus, thus only one possible conflict to resolve. 
[0031] Also, in situations where a large number of 
cnange logs have been generated, there is considerable 
potential inefficiency inherent in the old sequence of 
downloading and applying each change log in turn. In a 
given item entry tnere may only be one or two tields and 
there may be other fields belonging to the same item 
so scattered throughout entries in the sequence of change 
logs. Moreover, some of these later entries overwrite 
earlier ones. As a result, a separate transaction is re- 
quired for each occurrence of an item record in the 
change log sequence and it is possible for the same field 
>5 in a record to be updated many times during a single 
sync. With an aggregated change log redundant chang- 
es sro eliminated and records car, te updated rr.crs ef - 
ficiently via a single transaction. To further maximize the 
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efficiency of the present invention, a means is provided 
for compacting or compressing the aggregated change 
log if a compact threshold is ever exceeded. 
[0032] For items consisting of large quantities of data 
such as files (e.g., Microsoft Word® document), chang- 
es are transmitted as "binary deltas." These are opti- 
mized packages, which encode the information required 
to convert a binary image from one ■version" to the next. 

taken together, eventually exceeds the size of the object 
itself. At this point a new base version, or version "0," 
may be established and the complete object sent. How- 
ever, the change logs contain an identifier not the actual 
delta, or data. This identifier is used to request the delta 
itself, when the change log is applied. Thus, in aggre- 
gating change logs for files, if a new base version is en- 
countered existing changes can be flushed and only the 
new base and any subsequent changes need be down- 
loaded. Also, if a user has a conflict and resolves it by 
choosing a local version of the file, a new version n 0 n 
must be established, as there is no way of knowing the 
relationship between the local version and those in the 
stored change logs. 

[0033] The change log aggregation and optimization 
mechanism of the present invention is implemented in 
the client device. Specifically, the change log aggrega- 
tion and optimization mechanism of the present inven- 
tion is inserted into the download-and-apply sequence 
where change logs are fetched and interpreted to up- 
date application data. Instead of applying each change 
log as it is downloaded, it is merged into the "rolled" or 
aggregate log. The change log aggregation and optimi- 
zation mechanism of the present invention is further il- 
lustrated in the flow diagrams of Figures 1 A and 2. 
[0034] Figure 1A illustrates the addition of a current 
change log to an aggregate change log. The download 
and aggregate sequence begins in Step 100 of Figure 
1 A. A rolled log, or aggregate log, is created in Step 1 03. 
For each change log in the sequence, starting with Step 
1 06 the following steps are performed. In Step 1 09 the 
current change log is retrieved or downloaded. The cur- 
rent change log is then added to the aggregate log in 
Step 1 1 2. The current change log is then deleted in Step 
11 5 and Step 1 1 8 repeats Steps 1 09, 1 1 2, and 1 1 5 until 
there are no additional change logs in the sequence. 
Once this happens, the aggregate log is closed in step 
1 20 and the download and aggregate sequence is ter- 
minated in srep izi. !ne start Download and Aggre- 
gate sequence of the present invention can be summa- 
rized in the following steps: 

1 . Create aggregate log; 

2. Fetch (download) change log; 

3. Add change log to aggregate; 

4. Delete change log; 

5. Repeat the fetch add delete sequence for each 
change* !og to be aggregated; and 

6. Close aggregate log. 



[0035] Rgure 2 shows an expanded view of Step 1 1 2 
of Figure 1 A. For each item in the current change log 
the following steps are performed. The item in the cur- 
rent change log is first retrieved in Step 206 and the map 
s is updated in Step 209. The map stores meta-data to 
facilitate the aggregation or merging operation. It allows 
a one-to-many relationship between the keys and the 
items with which they are associated. This is a require- 
ment iur utjin types wnuseunanges are leaiizea as aei- 
10 tas," as each change must be applied separately and in 
- order. Thereafter, in Step 212 the aggregate log is up- 
dated and in Step 215 the item location information is 
also updated. Simultaneously in Steps 218 and 221 the 
current item retrieved from the current change log is writ- 
's ten to and stored in the aggregate log. In retrieving and 
merging subsequent change logs into the aggregate 
log, it is important to note that this process combines the 
data from two records for the same item using the meta- 
data in the map to locate each field and find its data in- 
20 side the correct change log. Once the item with its 
merged data has been written to the end of the aggre- 
gate log, the previous entry is marked obsolete, and can 
thus be deleted at a later time. 
[0036] Once there are no additional items in the cur- 
25 rent change log, in Step 227, the present invention de- 
termines whether the compact threshold has been ex- 
ceeded. If the compact threshold has been exceeded 
then, in Step 230, a compact aggregate log step 230 is 
initiated to compress or compact the size of the aggre- 
30 gate log. Compact aggregate log step 230 essentially 
removes obsolete records in the aggregate log to mini- 
mize the size of the aggregate log and thus conserve 
memory in storage servers 12, 14. This is accomplished 
by iterating over each record in the aggregate log, read- 
as jng only the valid records (having skipped over any ob- 
solete items), and writing the item back into the aggre- 
gate log without skipping obsolete items, effectively slid- 
ing data in the aggregate log back over obsolete records 
and thus eliminating them. 
40 [0037] Otherwise if the compact threshold in Step 227 
has not been exceeded then the sequence of steps in- 
itiated by Step 112 terminates in Step 233. The series 
of steps beginning with Step 1 12 effectively reads each 
record from the current change log and places this in- 
45 formation in the aggregate log. The Add Change Log to 
Aggregate Log sequence of Fig. 2 can be summarized 
in the following steps: 

1 . Get item information; 
so 2. Update the map; 

3. Update the aggregate log; 

4. Update item location information (in the map); 

5. Repeat the above sequence while we get a valid 
item; and 

55 6. Compact the aggregate log if the compact thresh- 
old has been reached. 

[0038] Figure 3 shows an expanded view of Step 206 
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- of Figure 2. In Step 303, the present invention first de- 
termines whether the current position is less than the 
size of the change log. If the current position of the 
record is not less than the size of the change log then 
an invalid message is returned in Step 339 and the proc- 
ess terminates in Step 342. However if the current po- 
sition is less than the size of the change log then an 
operation code is read in Step 306. The present inven- 
tion uici'i ucici i nil ico, hi oicrp oCa, wiieuitM me upeifctilun 

code is associated with a valid operation. If it is not a 
valid operation then a log error is generated in Step 318, 
an invalid message is returned in Step 339 and the proc- 
ess terminates with Step 342. If, however, the operation 
code is associated with a valid operation the current po- 
sition of the record is retrieved from the change log in 
Step 31 2. In Step 31 5, the invention determines whether 
the operation code corresponds to a "NOP," or a non- 
operation, in Step 315. The NOP essentially models a 
skip behavior. If the operation code corresponds to an 
NOP, a valid message is returned in Step 336 and the 
process terminates in Step 342. However, if the opera- 
tion code is not an NOP, information associated with the 
item is read from the change log "ID, Parent ID, flags, 
type" in Step 321. 

[0039] In Step 324, the invention determines whether 
the operation code corresponds to a delete function. If 
the operation code corresponds to a delete function then 
a valid message is returned in Step 336 and the process 
terminates in Step 342. However if the operation code 
does not correspond to a delete function, then while 
there is another field for the current item the field infor- 
mation for the item is obtained an added to the item in 
Step 330. The for-loop that begins in Step 327 ends in 
Step 333 when there are no additional fields for the cur- 
rent item. Thereafter, a valid message is returned in 
Step 336 and the process terminates in Step 342. The 
following steps summarize the "Getltemlnfo" sequence 
of Fig. 3: 

1 . If read position past end return invalid; 

2. Read operation code; 

3. If it is invalid return valid; 

4. Get current position; 

5. If operation is a "NOP" (no operation) return valid; 

6. Read item information; 

7. If operation is a delete return valid; 

8. Get field information and add to current item; 

9. Repeat get field and add Tor each field in the cur- 
rent item record; and 

1 0. Return valid. 

[0040] Figure 4 shows an expanded view of Step 209 
of Figure 2. At this point all the fields in a given item have 
been retrieved from the change log. In Step 403 the in- 
vention determines whether the item is in the map. If the 
item is not in the map then, in Step 409, the item is in- 
serted into the map, th3 itam and fteld location informa- 
tion is updated in Step 41 8 and the process terminates 



in Step 421 . However, if the item is in the map then, in 
Step 406, the invention determines whether the item re- 
trieved is a file. If the retrieved item is a file then, in Step 
412, a sequence of steps to execute and "add file item 

5 to map" is performed in Step 412. Thereafter the item in 
the field location information is updated and the process 
terminates in Step 421 . However, if the retrieved item is 
not a file then the sequence of steps to "default add item 
io map is penormea in &iep 41 a. Again, tne irem and 

10 field location information is updated and the process ter- 
minates in Step 421 . 

[0041 ] Figure 5 shows an expanded view of Step 41 2 
of Figure 4. In Step 503 the invention determines wheth- 
er the file has a new base version. If the file has a new 

15 base version then, in Step 515, the existing entries are 
removed from the map, the records in the aggregate log 
are marked obsolete, the file is inserted into the map 
and the process ends in Step 525. However, if the file 
does not have a new base version then the invention 

20 determines whether the file is a change, a force change, 
or a move in Step 506. If the file is either a change, a 
force change, or a move then, in Step 51 6 the file is add- 
ed to the map. File item changes must be applied serially 
and cannot be aggregated. This requires the use of a 

25 map that allows a "one to many 11 relationship between 
keys and items to maintain a "stack" of item records for 
a single item. The process then ends in Step 525. 
[0042] However, if the file item is not a change, a force 
change, or a move, then the invention determines 

30 whether the file item is a delete in Step 509. If it is a 
delete, then in Step 51 9 the existing entries are removed 
from the map, the records in the aggregate log are 
marked obsolete, the delete is inserted into the map, 
and the process ends in Step 525. If the file item does 

35 not have a new base version, is not a change a force 
change or a move and is not a delete, then by default in 
Step 512 ft is an add. In Step 522, the existing entries 
are removed from the map the records in the aggregate 
log are marked obsolete and the add is inserted into the 

40 map. Once this has been done, the process ends in Step 
525. 

[0043] Rgure 6 shows an expanded view of Step 41 5 
of Rgure 4. In Step 603 the invention determines wheth- 
er the item (in this case, the item is not a file) is a change, 

45 a force change, or a move. If the item is a change, a 
force change, or a move, then, in Step 61 2, the item from 
the new log is merged with the item in the aggregate 
role. However, if the item is not a change, a force 
change, or a move, then, in Step 606, the invention de- 

50 termines whether the item is an "add." If the item is an 
add, then in Step 615, the existing entries are removed 
from the map, the records in the aggregate role are 
marked as obsolete, and this add is inserted into the 
map. However, if the item is not an "add" then in Step 

55 609, by default the item is then a delete. Subsequently, 
in Step 618, the existing entries from the map are re- 
moved, L'iC- re-cords in Li te aggregate log arc* marked as 
obsolete, and the delete is inserted into the map. After 
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Steps 61 2, 61 5 and 61 8 the process ends with Step 621 . 
[0044] Figure 7 shows an expanded view of Step 612 
of Figure 6. Beginning with Step 703, for each field in 
the current item the following steps are performed. In 
Step 706, the present invention determines whether the 5 
current field is a body or a formatted body field. If the 
current field is not a body or a formatted body field then 
the invention determines, in Step 712, whether a current 

non-I anceiuj caiold. n lift? uunciti lltsiu dllttciuy t?AK>U> 

then the fields are merged in outside process 71 7. How- 10 
ever, if the current field does not already exist then, in 
Step 720, the current field is added to the item in the 
aggregate log. Turning back to Step 706, if the field is a 
body or a formatted body field then the invention next 
determines whether the field has already been purged 15 
in Step 709. If it has not been purged then, in Step 714 
the previous instance of either field, if any, are found and 
purged and the current field is subsequently added to 
the item in the aggregate log in Step 720. However, if 
the field has been purged then Step 709 proceeds di- 20 
rectfy to Step 720 where the current field is added to the 
item in the aggregate log. The for-loop beginning with 
Step 703 ends with Step 723 when the process has cy- 
cled through each field in the current item. The process 
then terminates in Step 726. & 
[0045] Figure 8 shows an expanded view of outside 
process 71 7 of Figure 7. In Step 803, the invention first 
determines whether a given field is a collection. If the 
field is a collection then, in Step 812, for each field in 
the current collection the following steps are performed. 30 
In Step 818, the invention determines whether the cur- 
rent field already exists. If the current field already exists 
then outside process 823 is performed and the field in 
the current collection is merged into the aggregate log. 
If the current field does not already exist then, in Step 35 
826 the current field is added to the list. Step 829 then 
repeats Steps 81 8, 823 and 826 until there are no addi- 
tional fields in the current collection. Going back to Step 
803, if the field is not a collection then the invention de- 
termines whether the field is a body or a formatted body *o 
field in Step 806. If the field is not a body or a formatted 
body field then, in Step 814, the field information is re- 
placed. However, if the field is a body or a formatted 
body field then, in Step 809, the invention determines 
whether the Held has already been purged. If the field *s 
has not already been purged then, in Step 81 6, the pre- 
vious instance of either field, if any, is found and purged, 
ir tne neia has oeen purged then, in Step 820 the current 
field is added to the item in the aggregate log. After 
Steps 829, 81 4, and Step 820, the process ends in Step so 
832. 

[0046] Figure 9 shows an expanded view of outside 
process 230 of Figure 2. Starting with Step 903, the in- 
vention first determines whether there are any obsolete 
records. If not, the process ends in Step 972. However,. 55 
if obsolete records exist then, in Step 906, a seek to the 
start of records ir. the aggregate- log ?or both read and 
write instances is performed. The for-loop initiated by 



Step 909 begins in Step 912 with the operation code for 
the read instance being read in. In step 921, the inven- 
tion determines whether the operation code read in Step 
912 is a no-op. If the operation is a no-op, the operation 
code is written in step 933. If after Step 933, the end of 
the aggregate log has been reached in Step 966, then 
the aggregate log size is reset and the aggregate log is 
flushed in Step 969 and thereafter the process ends in 
Siep 

[0047] However, if the operation in Step 921 is not a 
no-op then, in Step 918, item information comprising ID, 
Parent ID, type, and flags is read. In Step 915, so long 
as the current item is obsolete, the following steps are 
performed. In Step 924 the read pointer is advanced to 
the next operation code. In Step 936 the operation code 
is read and in Step 945 the invention determines wheth- 
er the operation is a no-op. If the operation is not a no- 
op then, in Step 954, item information comprising ID, 
Parent ID, type, and flags is read. However, if Step 945 
results in a no-op, then, in Step 960, the process loops 
back to Step 915 while the current item is obsolete. At 
the same time, a write operation is performed in Step 
927 and in Step 930, the invention determines whether 
this operation code is associated with a no-op. 
[0048] If the operation code of Step 927 is associated 
with a no-op, then the process proceeds to Step 966 
which then loops back to Step 909 unless the end of the 
aggregate log has been reached. However, if the oper- 
ation code of Step 927 is not associated with a no-op 
then in Step 942, item info (ID, Parent ID, type and flags) 
are written. If the operation code of Step 927 corre- 
sponds to a delete then the process proceeds to Step 
966, which then loops back to Step 909 unless the end 
of the aggregate log has been reached. If the operation 
code of Step 927 is not a delete then while there is an- 
other field for the current item the following steps, be- 
ginning with Step 943, are performed. In Step 948, the 
field is read and in Step 957 the field is written. Once 
there are no additional fields for the current item the 
process proceeds to Step 966 and proceeds to Steps 
969 and 972 if the end of the aggregate log has been 
reached. 

[0049] Figure 10 shows a flow chart illustrating the 
process for getting field information. The field tag and 
the current position are initially read in Step 1003. In 
Step 1 015 the invention determines whether the field is 
a collection. If the field is a collection then, in Step 1 021 , 
the collection field (count) is read and saved. Then, in 
Step 1 033, the field/count pair is added to the collection 
field stack. The process then terminates in Step 1066. 
However, turning back to Step 1 01 5, if the field is not a 
collection then, in Step 1006 the invention determines 
whether the field is a part of a collection. If the field is 
not part of a collection then, in Step 1048 the field is 
added to the current item's field list. The change log is 
then advanced to the next field in Step 1060 and the 
process terminates ai Slep 1066. Turning back tu Siep 
1 006, if the field is part of a collection then, in Step 1 009, 
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the field is saved on a holding stack. Thereafter in Step 
1 01 2, the following steps are performed until the collec- 
tion field stack is empty. In Step 1018 the invention de- 
termines whether the current collection is full. If the cur- 
rent collection is not full then, in Step 1 024, the invention 5 
determines whether the holding stack is empty. If the 
holding stack is empty the aggregate log is advanced to 
the next field as provided in Step 1 060 and the process 

diuo wiui oicfj iGuu: nuwcvci, n uit? nuiumg bmuiv i& 

not empty then, in Step 1030, the field is popped from w 
the holding stack. The field is then added to the current 
collection field's list and its counter is decremented in 2. 
Step 1045. 

[0050] Turning back to Step 1 01 8, if the current col- 
lection is full then, in Step 1027, the current collection 15 
is added to the holding stack and removed from the col- 
lection field stack Then in Step 1036, the invention de- 
termines whether the collection field stack is empty. If 
the collection f ield stack is empty then , in Step 1 051 , the 
field entry is popped off the holding stack and added to 20 
the parent item's field list until there are no more entries 
in the holding stack, so long there is an entry in the hold- 
ing stack (i.e., if the condition established by the for-loop 
defined by Steps 1042 and 1057 is still valid), Turning 
back to Step 1 036, if the collection field stack is not emp- 25 3. 
ty then, in Step 1039, the invention determines whether 
the current collection is full. If the current collection is 
not full then, in Step 1054, the field is popped off the 
holding stack, added to the current collection field, and 
the collection count is decremented. If the current col- 30 
lection is full, then Step 1 039 proceeds directly to Step 4. 
1 063 and Step 1 063 will loop back to Step 1012 unless 
the collection field stack is empty. 
[0051] The foregoing description of preferred embod- 
iments of the present invention has been provided for 35 
the purposes of illustration and description. It is not in- 5. 
tended to be exhaustive or to limit the invention to the 
precise forms disclosed. Many modifications and varia- 
tions will be apparent to practitioners skilled in this art. 
The described embodiments were chosen and de- 40 
scribed in order to best explain the principles of the in- 6. 
vention and its practical application to thereby enable 
others skilled in the art to understand the invention for 
various embodiments and with various modifications as 
are suited to the particular use contemplated. It is in- 45 
tended that the scope of the invention be defined by the 7. 
following claims and their equivalents. . 



said application data; 

adding said first change log to an aggregate 
log; 

deleting said first change log; 
repeating said downloading, adding, and delet- 
ing steps for a next change log of said plurality 
of change logs until no additional change logs 
exist; and 

applying saiii aggregate tog 10 saia appucaiion 
data to update said application data. 

The method of claim 1 , wherein said adding step 
further comprises the steps of: 

(a) retrieving information for a valid item in said 
application data; 

(b) updating a map of said aggregate log, said 
map storing meta-data; 

(c) writing said item to said aggregate log; 

(d) updating a location of said valid item in said 
map; and 

(e) repeating steps (a) - (d) for all remaining val- 
id items of a current change log. 

The method of claim 2, further comprising the step 
of: 

(f) compacting said aggregate log if a compact 
threshold is exceeded. 

The method of claim 1 , wherein said application da- 
ta comprises data classes for contacts, internet 
browser bookmarks, calendar events, email mes- 
sages, notes, tasks, and files. 

The method of claim 4, wherein said contacts com- 
prise records identifying names, addresses, phone 
numbers, and email addresses for a plurality of in- 
dividuals. 

The method of claim 4, wherein said files comprise 
word processor specific documents, electronic 
presentations, spreadsheets, and executable files 
in binary format 

The method of claim 1 , wherein said application da- 
ta is in a universal data format. 



8. An apparatus for updating application data in a cli- 
so ent device of a data transfer and synchronization 
system, said apparatus comprising: 



Claims 

1. A method for updating application data in a client- 
device of a data transfer and synchronization sys- 
tem, said method comprising the steps of: 

downloading a first change log of a plurality of 
change Jogs from a server system, each of said 
plurality of change logs reflecting changes to 



a downloading routine for iteratively retrieving 
a plurality of change logs from a server system, 
each of said plurality of change logs reflecting 
changes to said application data; 

a iVicrglfiy rouliris fo; .teraiivsly aggiegaftrigihc? 
contents of said plurality of change logs to an 
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aggregate log; 

a change log deletion routine for iteratively de- 
leting said plurality of change logs; and 
an updating routine for applying the contents of 
said aggregate log to said application data to 5 
update said application data. 

9. The apparatus of claim 8, wherein said merging rou- 
tine iuiLMt;i uufiipnses. 

an item retrieval routine for retrieving a plurality 
of items from said application data; 
a map update routine for updating a map of said 
aggregate log wherein said map stores meta- 
data; 

a field retrieval routine for retrieving field infor- 
mation from said plurality of items from said ap- 
plication data; and 

an item location update routing for updating the 
location of a plurality of items in said map. 

1 0. The apparatus of claim 9, wherein said merging rou- 
tine further comprises: 

an aggregate log compacting routine. 

11 . A method for aggregating the contents of accumu- 
lated change logs into an aggregate log and apply- 
ing said aggregate log to update application data in 
a first client device of a data transfer and synchro- so 
nization system, said method comprising the steps 
of: 

downloading a first change log of a plurality of 
change logs from a server system, each of said 35 
plurality of change logs reflecting changes to 
said application data; 

adding said first change log to an aggregate 
log; 

deleting said first change log; *° 
repeating said downloading, adding, and delet- 
ing steps for a next change log of said plurality 
of change logs until no additional change logs 
exist; and 

applying said aggregate log to said application 
data to update said application data. 
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